Written by Jim Calvin, Jeremy Nelson and Peter N Lewis.
What RumorMill Does
RumorMill is a news (NNTP) server. The News protocol is ideally suited to public discussion, and RumorMill is designed to make it easy to create (and remove) local or shared Newsgroups. RumorMill (and other NNTP) servers can easily be linked together to create a wide spanning communication network.
RumorMill is a server. To read news from a RumorMill Server you need a news client like Newswatcher, YA-Newswatcher, Netscape or Internet Explorer.
News and Usenet
Any discussion of News must include a discussion of Usenet. Usenet is a collection of 10,000 or so newsgroups, with about 1 Gigabyte of new articles posted per day. A full Usenet newsfeed requires a fast network connection, a fast computer and lots of hard disk space.
RumorMill can probably handle a full newsfeed, but it is designed to handle a partial newsfeed and to facilitate the creation of local newsgroups: newsgroups which are only present on the one news server, or distributed to a moderately small group of news servers.
Setting Up RumorMill
The RumorMill distribution comes with two separate applications, RumorMill and RumorMill Setup. RumorMill is the server, while RumorMill Setup is the configuration application. This separation of server and configuration software keeps the server application small, so it takes up less memory, and it also allows you to configure RumorMill remotely: RumorMill Setup can connect from another machine to allow convenient maintenance.
To setup RumorMill you need to specify what Newsgroups you wish to host, where you wish to gather news from and send it to (Newsfeeds), and who can use the service (Site Restrictions). RumorMill has a number of other settings which you can use to tighten security, reduce Spam and tune performance.
Where are you running RumorMill?
When you run RumorMill Setup you will be asked to specify the location of your RumorMill server. If it is running on the same machine as RumorMill Setup select On This Computer, otherwise type in the DNS Name (eg news.stairways.com.au) or IP number (eg 203.10.11.12) of the machine running the RumorMill server. If RumorMill is not running locally (on the same machine as RumorMill Setup) you will need to enter a Site Administrator’s Password. By default there is no password. RumorMill can not be configured remotely if there is no Site Administrator Password. (The password can be set in the Security window, using a copy of RumorMill Setup in local mode.)
Note: RumorMill Setup saves any password you enter for a remote server in the file RumorMill Setup Preferences in the Preferences folder.
Once you have retrieved the preferences you can modify or update the preferences on the server. The most important information you need to set is in the Newsgroups and Newsfeeds windows.
Newsgroups
A Newsgroup is a discussion area, a collection of articles posted by different users of the service. Newsgroup names are strings of normal letters and numbers, usually separated by dots (spaces are not allowed). The name should reflect the content and purpose of the newsgroup.
If you wish to receive Usenet news the group names must be the same as the Usenet news name. All groups you wish to collect must be listed explicitly in the Newsgroups window or RumorMill will not collect them.
Newsgroups are not explicitly marked as local or global: distribution of the Newsgroups is determined in the Newsfeeds window.
You can add newsgroups individually using the Add button, or you can add a list of Newsgroups using the Open Newsgroups file. A Newsgroups File must be either a plain text file (such as those created by SimpleText or BBEdit) or a Newswatcher preference file. If you are adding a large number of newsgroups it is easiest to list them in a text editor (such as BBEdit) and then load them into the Newsgroups window, or directly using the Create Groups... menu option in RumorMill.
Note that the Newsgroups window will not display more than 20 000 newsgroups. RumorMill Setup will still behave correctly, but it does not attempt to display more than 20 000 Newsgroups. For more information about handling large numbers of Newsgroups see the General Advice section below.
Newsfeeds
Newsfeeds are other NNTP servers where news articles come from, and are fed to. The Newsfeeds window lets you control whether news is fed to, received from or pulled from specific hosts. It also allows you to specify how often articles are fed to downstream newsfeeds, and how often they are pulled from upstream newsfeeds.
When you add a new newsfeed you are presented with a Newsfeed Entry window. Accept Articles From This Host determines whether RumorMill will let the newsfeed act as an upstream newsfeed. If this checkbox is not enabled RumorMill will reject attempts by this host to pass on articles.
The Active Pull feature makes RumorMill pro actively connect to the host and pull articles down (as if it were a client). Active Pull cannot be enabled unless Accept Articles From This Host is enabled. This is a useful feature if you cannot organize for the newsfeed to push feed news, or your RumorMill server is not on a permanent link.
Note that if you wish to host a large number of groups (more than a couple of hundred) you should probably arrange to be Push fed. When RumorMill pulls groups it intermittently checks with the Newsfeeds to see whether new articles have arrived. This causes unnecessary load on both RumorMill and the the Newsfeed. (And a significant amount of network traffic when large numbers of newsgroups are involved.)
Feed Articles To This Host determines whether RumorMill will attempt to pass on new articles to this host. If this is not checked new postings will not be fed to the host.
When you add a new newsgroup RumorMill Setup automatically assumes you want all newsgroups defined in the Newsgroups window Pulled. To change this default behaviour click on the Edit Group List button.
The Group List window allows you control over specific newsgroups on a newsfeed by newsfeed basis. On the left is a list of groups associated with the current Newsfeed, the Group List, and on the right is the Master List, a full list of groups as defined in the Newsgroups window.
You can ‘move’ groups into the Group List from the Master List using the ‘<<’ button. You can also add patterns to the group list which will match more than one group using the asterisk (‘*’) or question mark (‘?’). For example, to match all groups in the alt hierarchy you could use the pattern:
alt.*
The question mark matches single characters. So the pattern:
al?.???
Would match with ‘alt.sex’, ‘ala.aaa’ and ‘alt.six’, but not with ‘altt.aaa’, ‘alt.sixs’ or ‘blt.aaa’.
When you add a new group you can specify whether the server should Send the Group To the Newsfeeds, Exclude the Group From Being Received and Pull the Group From the Newsfeed. Note that this is a slightly different set of functions from the Newsfeed window, and it does not override the settings in the Newsfeed Entry window: if you do not have Pull enabled it the Newsfeed Entry window, no groups will be pulled even if they are marked as Pull in the Group List window.
Save the Preferences
That is all the configuration RumorMill requires. You can examine the Security and Site Restrictions configurations, but you should probably Save your preferences back to the server using the Save menu command or command-S. The Save As menu option lets you ‘transfer’ preferences to another server, or save a backup to disk.
Load and Save operations can take some time, especially when there is a large group list. Once the group list is loaded RumorMill Setup only sends changes to the Group List to RumorMill. It does not attempt to ascertain whether the settings on the RumorMill Server have changed when it saves them, so sometimes RumorMill and RumorMill Setup will be out of step. Reload the preferences from RumorMill to update RumorMill Setup.
Site Restrictions
The Site Restrictions window lets you restrict what machines have access to the server. Computers are restricted on the basis of their IP number. The server comes with a default mask permitting access by computers in the same subnet as the server (that is, the first three numbers in the IP number must match the server’s).
For instance, if the server is at the IP number 1.2.3.4, any client coming from a machine with IP numbers between 1.2.3.1 and 1.2.3.255 can connect to the server. A computer with IP number 5.6.7.8 would not be permitted to connect. The IP mask for this configuration is 1.2.3.4/255.255.255.0. (The mask makes a lot more sense if you consider it as a binary number: 255 is 11111111 in binary, 0 is 00000000. 255 means match all bits, 0 means match no bits.)
More complex masks can be used like, 127.2.3.4/1.255.0.255, which would match all IP numbers between 1.2.1.4 and 1.2.255.4, 3.2.1.4 and 3.2.255.4, 5.2.1.4 and 5.2.1.4 and so on (that is the first number is odd and the remaining three are between .2.1.4 and .2.255.4). The ordering of the IP masks is important: RumorMill starts at the top of the list and stops as soon as it finds a mask which specifically permits or denies the IP number. If it reaches the end of the list without specific reference the IP number is denied. Generally speaking more specific masks should be placed above more general masks.
RumorMill also adds a mask for each of the servers defined in the Newsfeed window (otherwise they would not be able to feed articles to RumorMill, and RumorMill would not feed articles to the Newsfeed). There can be no more than 20 Site Restrictions masks.
The Site Restrictions window also allows you to check whether a particular IP number is permitted or denied using the Test IP number, which tests against the current list of masks.
Note that it is not possible to check against DNS names (like stairways.com) and use these to prohibit access because DNS names can be ‘spoofed’ by other machines (for instance, other machines could pretend to be stairways.com), and thus DNS names are not secure way of restricting access.
Security
The Security Window allows you control over whether Posting is Allowed and whether Rejected Messages are Logged. Messages are usually rejected for some kind of formatting error. Rejected messages are appended to a file called “Rejected Articles” when they are being logged. There is probably no reason to enable this feature unless you think the server is misbehaving.
Administrator OK’s New Groups and Administrator OK’s Group Deletes lets you have some control over the creation and deletion of groups. For an explanation of these controls see the General Advice section. These features are not currently fully supported in RumorMill Setup.
The Site Administrator password is only required when conducting remote configuration of a RumorMill Server. If the Site Administrator password has not been set it is not possible to administer RumorMill remotely. When a password is set in the Security window it is not automatically set as the password for the current remote server (otherwise you could never change the password). You need to enter the new password in the Server Location window after updating the password. Note that the password in the Server Location window is stored locally in the file RumorMill Setup Preferences in the Preferences folder (as is the SIVC options in the Settings window).
Settings
The settings window contains miscellaneous settings, mostly to do with the performance characteristics of your RumorMill server.
Expire Articles Time: Expiring articles is processor intensive, so it can scheduled away from peak periods of server usage (usually sometime late at night).
Hold Articles: When an article is fed to the RumorMill Server without an expiry period, RumorMill will expire it (delete it) after holding it for this period of time.
Maximum Number of Clients: Permits you to restrict usage of the RumorMill server, thereby constraining its impact on the server and your network.
Maintain minimum free disk space: Is a suggestion to RumorMill of how much disk space it should make sure is available on the volume to which it is pulling articles. This prevents it from filling the hard disk and causing other applications to crash. Note that RumorMill can easily over run this number since it does not check the free disk space while downloading an article- it only checks between article transfers.
Help Note: Is only seen by NNTP Clients, it should not be seen by anyone reading articles. You should include an E-Mail address for other news administrators to contact in case of problems.
Rebuilding the Database
RumorMill uses a database of news articles to increase speed and reduce disk usage. Occasionally this database will become corrupted (due, for instance, to a power outage). RumorMill usually autodetects when a database becomes corrupt and will rebuild the index information for the database. You can also force RumorMill to rebuild its database information by holding down control and option while launching the program.
The rebuild process is, unfortunately, not well behaved at this time. (This regrettable behaviour will be corrected in the next release. ) During a rebuild RumorMill will seize your machine and not give it back until it has finished. Note, a full database rebuild can take some hours: you may need to organize to rebuild the database overnight.
General Advice
This section discusses some specific uses of RumorMill.
Propagating Articles Downstream
When a user posts a new message to RumorMill, the RumorMill will look through its list of Newsfeeds to see which servers it should attempt to pass the article on to. If those servers are not configured to permit RumorMill as an upstream newsfeed, they will reject the article.
You will need to ask the administrator of the downstream server to add your RumorMill server to the list of accepted servers. Note, that this is usually achieved by checking incoming connections against a list of server IP addresses. If you do not have a fixed IP address, the downstream server will probably not be able confirm RumorMill as a server. Thus it is generally difficult to propagate articles downstream using a Dial-Up connection.
Big Group Lists, Lots of News
If you carry a significant number of Newsgroups you should make sure you have plenty of disk space and a good connection. In our testing we found that about 60 (unexceptional) newsgroups took up 42 Mb of disk space, built up over a couple of days through a 28.8K (permanent) link. In one test, RumorMill built a database of over 200MB in less than 24hrs handling a mere 770 groups (there are probably in excess of 10,000 active USENET newsgroups). UNIX based NNTP servers that support “all” active USENET newsgroups typically require a few gigabytes of disk space and a permanent T1 link. RumorMill may be able to handle a full Newsfeed if it is running on a fast machine with a good (fast/wide SCSI) hard disk. We have not tested it with a full newsfeed.
RumorMill Setup should handle fairly large lists of Newsgroups, but it will be slow. I recommend using the Menu commands in RumorMill (Create Groups... and Save Groups...) if you intend to have a large number of newsgroups.
You will probably need a static IP address so that your newsfeed can be configured to send articles to RumorMill (make sure you have Accept Articles from this Host set in the Newsfeeds window). Pulling the Newsgroups takes a lot of time since RumorMill has to check each newsgroup to see if there are any new articles, every time. With a Push feed from an upstream Newsfeed, the articles are simply sent to RumorMill when they arrive.
Phone Connections
RumorMill can be used on a dial-up (intermittent) connection, to pull news for an individual or group. RumorMill has a menu option (Start Pull) which can be used to trigger the Pull after connecting. Note, depending on your network configuration you may need to quit and restart RumorMill once you have connected.
Note also the section above on Propagating Articles Downstream.
Local Newsgroups
Local Newsgroups are easy to make: give them a name which is not currently used on any other Newsfeed and it will not be propagated. Just to make sure no external news articles are fed into the local newsgroup you should probably add is as an Excluded group in the Group List of all the Newsfeeds.
Advanced Functions
RumorMill supports a number of features which are not fully supported in RumorMill Setup. It is possible to access these features using a telnet client.
For instance RumorMill Setup does not currently support much Administrative control over the automatic addition and removal of newsgroups. (This will be supported at a later time.)
Speaking Directly to RumorMill
It is possible to telnet directly RumorMill to configure commands not currently supported by RumorMill Setup. This section will not cover the commands: more information is available on our web site.
We recommend using Nifty Telnet (or the like) rather than NCSA Telnet because NCSA Telnet sends a series of characters when it opens a connection which will cause the first command to return an error.
To connect to the RumorMill server, telnet to port 119. (In Nifty Telnet simply enter my.host.name:119 as the hostname, substituting the appropriate DNS name.) If you are connecting remotely you will need to enter the ‘xpass mypass’ command, where mypass is the password you have previously configured by connecting to RumorMill locally (either using a telnet client or RumorMill Setup).
After that you can issue the ‘help’ command and explore. Most preferences are listed using ‘xpref list’ or ‘xpref get’, and are set using ‘xpref set’. See the RumorMill web site for more information.
Group Creation and Deletion
Articles can have headers like "Control: " and "Also-control: " in them. These header lines can contain directives to create (newgroup) and delete (rmgroup) newsgroups. The "Admin Ok's" check boxes determine whether these control directives are executed when the articles are received, or deferred for the administrator to review and control the selective execution of the directives found in those articles.
Currently, in the deferred mode (Admin OKs), there is no way to view the messages from RumorMill Setup; they can only be viewed and executed using the "Execute deferred msgs..." command in RumorMill.
Glossary
article n. A posting to a newsgroup. Articles following a common subject are said to be in the same thread [of conversation].
cross-post [Usenet] vi. To post a single article simultaneously to several newsgroups. Distinguished from posting the article repeatedly, once to each newsgroup, which causes people to see it multiple times (which is very bad form). Gratuitous cross-posting without a Followup-To line directing responses to a single followup group is frowned upon, as it tends to cause followup articles to go to inappropriate newsgroups when people respond to only one part of the original posting.
DNS name n. Literally Domain Name Server name. A DNS name usually identifies a single computer or service- it is like a Post Office box in the real world. DNS names are mapped by Domain Name Servers to an IP address which is then used to route information to that computer.
downstream n. A news server which is downstream from your news server is fed articles from your news server, ie it is one of the places to which your news server sends articles. Two news server can be both upstream and downstream relative to each other.
host n. Another server on the network. For instance a ‘news host’ is computer on the network which provides news (NNTP) services.
IP number n. An Internet Protocol number uniquely identifies a computer somewhere in the world. IP numbers are used to route information to a computer and identify them the computer on the network.
A computer can have multiple IP numbers but it is relatively rare to from multiple computers to have the same IP number. (An exception is when there is a Redundant Array of Inexpensive Computers which can service heavy loads through a single entry point.) A DNS (Domain Name Server) maps DNS names to IP numbers.
newsfeed n. Another NNTP server which feeds articles to, or receives articles from other NNTP servers. Often refers to a source of Usenet news. Newsfeed, ‘news server’ and host can all be used interchangeably in the correct context.
newsgroup n. [Usenet] One of Usenet's huge collection of topic groups or forums. Usenet groups can be `unmoderated' (anyone can post) or `moderated' (submissions are automatically directed to a moderator, who edits or filters and then posts the results). Some newsgroups have parallel mailing lists for Internet people with no netnews access, with postings to the group automatically propagated to the list and vice versa. Some moderated groups (especially those which are actually gatewayed Internet mailing lists) are distributed as `digests', with groups of postings periodically collected into a single large posting with an index.
NNTP n. Network News Transfer Protocol. This is the protocol that servers (such as RumorMill) use to talk to each other, and clients (such as Newswatcher) use to retrieve and post articles. NNTP is plain-text and human readable, even if it is a little cryptic at times.
push feed n. Large newsfeeds are usually propagated by push feeding. When a new article arrives on a news server (either by someone posting to the news server or another news server feeding it an article) it can propagate the articles by looking up a list of other news servers it is meant to transfer articles to, connecting to them, and telling them the new article has arrived. RumorMill supports push feeding. See also pull.
pull v. RumorMill can act like a client, connect to another NNTP server and pull articles form the news server.
This is also known in the community as a "suck feed." It's weird, but apparently some folks think of "pull" as an inefficient implementation while a "suck" is efficient. This must date back to someone's implementation that did "pull" badly and another one that did "suck" well (or at least better). I must say that I prefer the word "pull" to "suck" ...
See also push feed.
server n. A server is a computer or piece of software which provides a service. Usually humans use servers through another piece of software generically called a ‘client’. For instance RumorMill is a news server, and to read news of a RumorMill you use a news client (like Newswatcher). Web Servers provide web services, which a human can use through a web client (like Netscape).
spam n. An article which is posted to a large number of news groups, usually with little relevance to the most of the newsgroups. Typical spam postings are adverts and Make Money Fast pyramid schemes. Spam is usually more of an annoyance than a problem, but it should definitely be discouraged.
subject n. A single line which should (but often doesn’t) summarise the topic of an article in a newsgroup.
thread n. As in a ‘thread of conversation’. A thread is usually a series of articles in a newsgroup with the same subject line or topic.
upstream n. A news server which is upstream from your news server feeds articles to your news server, ie it is one of the places from which your news server receives articles. Two news server can be both upstream and downstream relative to each other.
Usenet n. [from `Users' Network'; the original spelling was USENET, but the mixed-case form is now widely preferred] Usenet was originally created in the late 1970s as a "poor man's ARPAnet", to distribute news about the Unix Operating System. It has since grown to include over 10,000 separate newsgroups about many different topics. Some newsgroups are "moderated" so that messages have to be approved before anyone can read them, but most newsgroups are unmoderated. In an unmoderated newsgroup, anyone can place messages, and anyone can read them. Most messages are replies to other messages, and thus an endless discussion is formed. Posting to a newsgroup is not unlike writing email.
The Internet is the main medium for the distribution of Usenet News, but it wasn't always so. UUCP used to be the most common way to transfer Usenet, and it is still in use today. All that is required to run a news server is some computer hardware and a "Usenet Feed". Getting a Usenet feed is as simple as asking an existing Usenet site. Any site can feed or be fed by any other site, and thus it is practically impossible to control.
Registering
This program is Shareware, which means if you use it, you must pay for it. A single user license costs US$35.
You can pay in one of two ways: on-line registration using a web browser, or off-line registration using the Register program.
On-Line Registration
Our online registration can be found at:
<http://order.kagi.com/cgi-bin/register1.cgi?PL>
Off-Line Registration
Or, using the Register program, you need to:
1. Get hold of a copy of the Register program: Register comes with the RumorMill distribution. You can also get Register from the following sites:
<ftp://ftp.stairways.com/>
<ftp://mirrors.aol.com/pub/peterlewis/>
<ftp://ftp.amug.org/pub/peterlewis/>
..or there are download links on the following Web page:
<http://www.stairways.com/register/topay.html>
2. Run the Register program and fill out the form: You need to enter your name, email, postal address, and the shareware you wish to pay for. The form accepts many different payment methods such as: US Check, Money Order, Cash (in many different currencies), Visa, Mastercard, American Express, First Virtual, and Invoice (to be given to your accounts payable department).
3. Send it to Kagi Shareware: Then either email the data generated by the registration program or print it and send it via postal mail or fax. Credit card information is encoded by the Register program.
The address to send the completed form is output by Register when you Print or Copy the completed form. The addresses are:
Email: shareware@kagi.com
FAX: +1 510 652 6589
Snail-mail:
Kagi Shareware
1442-A Walnut Street #392-PL
Berkeley, California, 94709-1405
USA
You may distribute this program any way you wish as long as you don’t charge for it (reasonable download costs such as Compu$erve are ok (although who would call Compu$erve’s download costs reasonable?)). You must distribute the package in its entirety. We don’t guarantee any support, but we always answer our Email. If we don’t answer Email it is because your message didn’t get to us, or our reply bounced, so please try again and include a valid Internet address if you can.
You MAY NOT DISTRIBUTE this program on any disk or CD without our explicit permission.
Australians may pay in Australian dollars direct to us if they prefer.
Site Licensing
World-wide license: US$2000
Universities or companies site license: US$500
Curtin University and the University of Western Australia are exempt.
A site license covers usage of RumorMill on an unlimited number of machines within 100 miles of some arbitrary central point which are owned by the licensed organization. (A site license will not be useful unless you intend to run more than 14 copies of RumorMill.)
World Wide licenses remove the 100 mile radius restriction.
Warranty
This program should do what is described in this document. If it doesn’t, you can simply stop using it. If you paid for the product, and within a year find that it doesn’t do what has been described here, then you can notify Stairways Shareware and your money will be refunded and your license cancelled.
Fine Print
Peter Lewis hereby disclaims all warranties relating to this software, whether express or implied, including without limitation any implied warranties of merchantability or fitness for a particular purpose. Peter Lewis will not be liable for any special, incidental, consequential, indirect or similar damages due to loss of data or any other reason, even if Peter Lewis or an agent of his has been advised of the possibility of such damages. In no event shall Peter Lewis be liable for any damages, regardless of the form of the claim. The person using the software bears all risk as to the quality and performance of the software.
US Government
Government End Users: If you are acquiring the Software and fonts on behalf of any unit or agency of the United States Government, the following provisions apply. The Government agrees:
(i) if the Software and fonts are supplied to the Department of Defence (DoD), the Software and fonts are classified as "Commercial Computer Software" and the Government is acquiring only "restricted rights" in the Software, its documentation and fonts as that term is defined in Clause 252.227-7013(c)(1) of the DFARS; and
(ii) if the Software and fonts are supplied to any unit or agency of the United States Government other than DoD, the Government's rights in the Software, its documentation and fonts will be as defined in Clause 52.227-19(c)(2) of the FAR or, in the case of NASA, in Clause 18-52.227-86(d) of the NASA Supplement to the FAR.
Acknowledgements
Jim’s Acknowledgements
Thanks to Ellen for her patience and support while I worked on RumorMill. Thanks to Peter for the opportunity and to Jeremy for chasing too many bugs.
Jim Calvin, author of RumorMill.
Jem’s Acknowledgements
Thanks to Peter for his patience and support. Thanks to Jim for putting up with my amateur efforts: I hope people won’t be put off by RumorMill Setup!
